Robotics pertains to automated machines that take the place of humans in a variety of applications, for example, medical, manufacturing, and military. Typically, robots are guided by a computer program or electronic circuitry to provide control, sensory feedback, and information processing and resemble humans in appearance, behavior, and/or cognition.
Most robots include end of arm tooling (EOAT), otherwise known as end effectors, that interact with the work environment. End effectors may include impactive grippers such as jaws or claws that grasp an object by direct impact. End effectors may also include ingressive grippers such as pins or needles that penetrate the object such as that used in textile manufacturing. Furthermore, end effectors may include astrictive grippers that apply suction forces to the object.
Grasping by EOAT has been studied from various perspectives including planning, control, and learning. In real-world grasping, the full 3D shape of the object is hard to perceive and certain grippers are difficult to replicate with a physical model for study and grasp planning.
Several approaches have been successful in solving the problem of robotic grasping. For example, if the kinematics of the EOAT are known and a two-dimensional (2D) or three-dimensional (3D) model of the object is available, methods that consider form closure and force closure can be used to plan a grasp. Heuristic rules have been used to generate and evaluate grasps for three-fingered hands by assuming that the objects are made of basic shapes such as spheres, boxes, cones and cylinders, each with pre-computed grasp primitives. Other methods focus on grasping 2D planar objects using edges and contours to determine form closure and force closure. Further methods considered grasping planar objects by classifying them into a few basic shapes, and then used pre-scripted rules based on fuzzy logic to predict the grasp. Yet other methods used support vector machines to estimate the quality of a grasp given a number of features based on spin images. Closed loop feedback has also been used to perform grasps, both with visual feedback and tactile feedback. Most of these methods however assume a physical model of the gripper and often a very detailed physical model is required. In instances when the object geometry is known as well as the gripper geometry, control and planning algorithms are designed from the known geometric information for successful grasping with force closure and form closure.
Using learning algorithms to predict grasps have been used to provide a degree of generalization to grasping, thus making it applicable to previously unseen objects, as well as making it possible to add more features (as well as data) in order to increase the performance of the algorithm. It has been shown that a “grasping point” (where to grasp) can be estimated from an image of the object using a learning algorithm, and that the estimated grasping point can be generalized to a large number of novel objects. However, other parameters such as gripper orientation are not included and left to be estimated by other learning techniques.
Depth information, such as point cloud, has also been included to obtain higher performance. In addition, a segmented point cloud has been used to enable grasping in cluttered environments. In fact, learning algorithms have also been successfully applied to other object handling tasks such as placing an object in an unstructured environment and opening doors by turning door handles. These learning approaches show the possibility of interaction with the object without knowing the object geometry. However, these learning approaches are sub-optimal in that they only partly represent the gripper configuration by using low-dimensional representations such as a grasping point or pair of points.
There is a demand for predicting successful grasps for EOAT without requiring a physical model. Specifically, there is a demand for predicting the 3D location, 3D orientation and opening width or area of contact for an EOAT. The invention satisfies this demand.